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Abstract 

This note studies structural aspects concerning Optimal Positional Strategies (OPSs) in Mean 
Payoff Games (MPGs), it’s a contribution to understanding the relationship between OPSs in 
MPGs and Small Energy-Progress Measures (SEPMs) in reweighted Energy Games (EGs). Eirstly, 
it is observed that the space of all OPSs, optpE^, admits a unique complete decomposition 
in terms of so-called ejrfremal-SEPMs in reweighted EGs; this points out what we called the 
“Energy-Lattice of optpE|^’’. Secondly, it is offered a pseudo-polynomial total-time re¬ 
cursive procedure for enumerating (w/o repetitions) all the elements of ^ and for computing 
the corresponding partitioning of optpEo^. It is observed that the corresponding recursion tree 
defines an additional lattice ,^p, whose elements are certain subgames E' C E that we call ba¬ 
sic subgames. The extremal-SEPMs of a given MPG E coincide with the least-SEPMs of the 
basic subgames of E; so, is the energy-lattice comprising all and only the /easf-SEPMs of 
the basic subgames of E. The complexity of the proposed enumeration for both ,^p and 3^y 
is 6>(|Ep|£’|VT|.^^p|) total time and GdEHEl)-f ©(lEllc^pl) working space. Einally, it is con¬ 
structed an MPG E for which \£^y\ > |.^*|, this proves that ^^y ^nd S(Jy are not isomorphic. 

Keywords: Mean Payoff Games, Optimal Strategy Synthesis, Pseudo-Polynomial Time, 

Energy Games, Small Energy-Progress Measures. 


1. Introduction 


A Mean Payoff Game (MPG) is a two-player infinite game E = {V,E,w,{Vo,Vi)), that is 
played on a finite weighted directed graph, denoted = {V,E,w), where w.E^X, the vertices 
of which are partitioned into two classes, Vq and Vi, according to the player to which they belong. 

At the beginning of the game a pebble is placed on some vertex S V, then the two players, 
named Player 0 and Player 1, move it along the arcs ad infinitum. Assuming the pebble is cur¬ 
rently on some v € Vo, then Player 0 chooses an arc (v, v') € E going out of v and moves the pebble 
to the destination vertex v'. Similarly, if the pebble is currently on some v S Vi, it is Player 1 ’s turn 
to choose an outgoing arc. The infinite sequence v, v'... of all the encountered vertices forms a 
play. In order to play well. Player 0 wants to maximize the limit inferior of the long-run average 
weight of the traversed arcs, i.e., to maximize limi nfn^oo ^ tv(Vi', Vi+i), wh ereas Player 1 




). lEhrenfeucht and Mvcielskil ( 19791) proved 


wants to minimize the lim sup„^„ 

that each vertex v admits a value, denoted val^(v), that each player can secure by means of a 
memoryless (or positional) strategy, i.e., one depending only on the current vertex position and 
not on the previous choices. 
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Solving an MPG consists in computing the values of all vertices (Value Problem) and, for 
each player, a positional strategy that secures such values to tha t player (Optimal Strategy Syn¬ 
thesis). The corresponding decision pr oblem lies in N P f l coN P (IZwick and PatersonLll996h and 
it was later shown to be in UP fl coUP ( Jurdzihsd 1998 ). 

The problem of devising efficient algorithms f or solving MPGs has been studied extensively 
in the literature. The first milestone was settled in iGurvich et al.l (Il988h . in which it was offered 
an exponent ial time algorithm for solvi ng a slightly wider class of MPGs called Cyclic Games. 
Afterwards, IZwick and PatersonI (Il996h devised the hrst deterministic procedure for computing 
values in MPGs, and optimal strategies securing them, within a pseudo-polynomial time and 
polynomial space. In particular, it was established an GdypIfsIlT) upper bound for the time 
complexity of the Value Problem, as well as an upper bound of (9(|yl^lisIVTlogdfil/ly|)) for 
that of Optimal Strategy Synthesis ( Zwick and Paters^ 1 19961) . 

Several research efforts have been spent in studying quantitative extensions of i nhnite games 
for modeling quantitative aspects of reactive systems, e.g., the Energy Games (EGs) (IChakrabarti et al 
2003 : Bouver et al. . 2008 : Brim et all 2011 ). These studies unveiled interesting connections be¬ 
tween EGs and MPGs; and by relying on these techniques, recently the worst-cast time com¬ 
plexity of the Value Probl em and Optimal Strategy Synth esis was given an improved pseudo¬ 
polynomial upper bound ( Comin and Rizzil 2015, 2016al) : those works focused on offering a 
simple proof of the improved upper bound. However, the running time of the proposed al¬ 
gorithm turned out to be also OdVl^lEliy), the actual time complexity being ©(jyl^liiliy-f 
deg]-(v) •fp(v)), where fp(v) < dy| — l)|y|iy denotes the total number of times that a 
certain ener gy-lifting operator v) is applied to any v G y. A way to overcome this issue 
was found in icomin and Rizzl ( 2016bh . where a novel algorithmic scheme, named Jumping, was 
introduced; by tackling on some further regularities of the problem, the estimate on the pseudo¬ 
polynomial time complexity of MPGs was reduced to: GdEj log |y |) + ©(Lvev 
where, for every v G y, f'p(v) is the total number of applications of 5(-,v) tha t are made by the 


algorith m; fj- < (|y| — lj|y|iy (worst-case, but experimentally £j- <C see IComin and Ri^ 
( 2016bl) l. and the working space is 0(|y | -f lEl). With this, the pseudo-polynomiality was con- 
hned to depend solely on the total number ij- of required energy-liftings. 


Contribution. This work studies the relationship between Optimal Positional Strategies (OPSs) 
in MPGs and Small Energy-Progress Measures (S EPMs) in reweighted EG s. Actually this paper 


is an extended and revised version of Section 5 in Comin and Rizzi ( 20L5h . Here, we offer: 


1. An Energy-Lattice Decomposition of the Space of Optimal Positional Strategies in MPGs. 

Let’s denote by opt^Ej^ the sp ace of all the optimal posi t ional st rategies in a given MPG E. What 
allows the algorithms given in Comin and Rizzil ( 2015 . 2016allb ) to compute at least one cJq G 
optpE^ is a compatibility relation that links optimal arcs in MPGs to arcs that are compatible 
w.r.t. least-SEPMs in reweighted EGs. The family of all SEPMs of a given EG F forms a 
complete finite lattice, the Energy-Lattice of the EG F. Firstly, we observe that even though 
compatibility w.r.t. least-SEPMs in reweighted EGs implies optimality of positional strategies 
in MPGs (see Theorem O, the converse doesn’t hold generally (see Proposition |5]l. Thus a 
natural question was whether compatibility w.r.t. SEPMs was really appropriate to capture (e.g., 
to provide a recursive enumeration of) the whole optpEj^ and not just a proper subset of it. 
Partially motivated by this question we explored on the relationship between optpEj^ and (Sp. 
In Theorem m it is observed a unique complete decomposition of optpE^ which is expressed 
in terms of so called extremal-SEPMs in reweighted EGs. This points out what we called the 
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“Energy-Lattice associated to optpE^”, the family of all the extremal-SEPMs of a given 
MPG r. So, compatibility w.r.t. SEPMs actually turns out to be appropriate for constructing 
the whole opt^Ej^; but an entire lattice ^y of extremal-SEPMs then arises (and not just the 
least-SEPM, which turns out to account only for the join/top component of optpEj^). 

2. A Recursive Enumeration of Extremal-SEPMs and Optimal Positional Strategies in MPGs. 


It is offered a pseudo-polynomial total time recursive procedure for enumerating (w/o repetitions) 
all the elements of and for computing the associated partitioning of optpEp^. This shows 
th at the above ment i oned co mpatibility relation is appropriate so to extend the algorithm given 


Comin and Rizzil ( 2016bl) . recursively, in order to compute the whole optpEj^ and SEy ■ It is 


observed that the corresponding recursion tree actually dehnes an additional lattice SSy, whose 
elements are certain subgames E' C E that we call basic subgames. The extremal-SEPMs of 
a given E coincide with the least-SEPMs of the basic subgames of E; so, is the energy- 
lattice comprising all and only the Zeasf-SEPMs of the basic subgames of E. The total time 
complexity of the proposed enumeration for both t^Y ^y D’ works in 

space OdyllEl)-|-0(|£'||,^p|). An example of MPG E for which |,^p| > I S^Y I ends this paper. 

Organization. The following Section|2]introduces some notation and provides the required back¬ 
ground on inhnite 2-player pebble games and related algorithmic results. In Section [2 a suit¬ 
able relation between values, optima l strategies, and certain reweighting operations is recalled 
from Comin and Rizzil (12015 , 2016ah . Section|4]offers a unique and complete energy-lattice de¬ 
composition of optpE^. Einally, Section |5] provides a recursive enumeration of S^y and the 
corresponding partitioning of optpE^. 


2. Notation and Preliminaries 

We denote by N, Z, Q the set of natural, integer, and rational numbers. It will be sufficient to 
consider integral intervals, e.g., [a,b] = {z G \ a < z < b} and [a,b) = {z € Z \ a < z < b} for 
any a,b G Z. Our graphs are directed and weighted on the arcs; thus, if G = (V,E,w) is a graph, 
then every arc e € £ is a triplet e = (u,v,yve), where Wg = w(u,v) G Z. Let W = maxgg^ \we \ be 
the maximum absolute weight. Given a vertex u GV, the set of its successors is Ap“‘(m) = {v G 
V I {u,v) G £}, and the set of its predecessors is Ny(u) = {v GV \ {v,u) G £}. Let degp(v) = 
|Np (v)| -I- |Ap“^(v)|. A path is a sequence vqvi ... v„ ... such that (v,-, v,+i) G E. Let V* 
be the set of all (possibly empty) hnite paths. A simple path is a hnite path vqvi ... v„ having 
no repetitions, i.e., for any i,] G [0,n] it holds v,- f vj if i f j. A cycle is a path vqVi ... 
such that Vo • • • v„-i is simple and v„ = vq. The average weight of a cycle vq .. .v„ is w(C)/|C| = 
i w(v,', v,+i). A cycle C = vqVi ... v„ is reachable from v in G if there is some path p in G 
such that pnC ^ 0. 

An arena is a tuple E = {V,E,w, (Vb,yi)) where G'" = {y,E,w) is a hnite weighted directed 
graph and (yo,yi) is a partition of y into the set Vq of vertices owned by Player 0, and Vi owned 
by Player 1. It is assumed that G^ has no sink, i.e., V'’‘=^Ap“*(v) f 0; we remark that G^ is 
not required to be a bipartite graph on colour classes yo and Vi. A subarena E' (or subgame) 
of E is any arena E' = (y',E',w', {Vq,V{)) such that: V' C V, = V' HVi, E' C E, and 

w' = We- Given S GV, the subarena of E induced by S is denoted E|^, its vertex set is S 
and its edge set is £' = { (m, v) G E \ u,v G S}. A game on E is played for inhnitely many rounds 
by two players moving a pebble along the arcs of G^. At the beginning of the game the pebble 
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is found on some vertex G V, which is called the starting position of the game. At each turn, 
assuming the pebble is currently on a vertex v G V, (for i = 0,1), Player i chooses an arc (v, v') G E 
and then the next turn starts with the pebble on v'. Below, Fig. [T]depicts an example arena Fex. 



Figure 1; An arena Fex = (V, E,w, (VbjVi)). Flere, V = 

{A,B,C,D,E,F,G} and E = {(A,B,+3),(B,C,+3),(C,D,-5), 

(D,A-5),(E,A,0),(E,C,0),(E,F,0),(E,G,0),(F,G,-5),(G,F,+3)}. Also, Vq = 

{B,D,E,G} is colored in red, while Vi = {A,C,F} is filled in blue. 


A play is any infinite path vqvi .. .v„. .. G V“ in F. For any i G {0,1}, a strategy of Player i 
is any function <7, : V* x Vi ^ V such that for every finite path p'v in G^, where p' G V* and 
V G Vi, it holds that (v, (7j(p', v)) G £. A strategy CJ, of Player i is positional (or memoryless) if 
(yi{p,Vn) = v'„) for every finite paths pv„ = vq ... v„-iv„ and p'v'„^ = Vg... in 

such that v„ = G V,. The set of all the positional strategies of Player i is denoted by . A 
play VQVi... v„ ... is consistent with a strategy a G E; if vy+i = <7(vovi ■■■Vj) whenever vj G V). 

Given a starting position G V, the outcome of two strategies Oo G and (7i G Ei, denoted 
outcome^(Vi, 007 <7i), is the unique play that starts at and is consistent with both Oo and (7i. 

Given a memoryless strategy O’, G of Player i in F, then G(o,,r) = {V,Eaj,w) is the graph 
obtained from G^ by removing all the arcs (v, v') G E such that v G V) and v' O'i'(v); we say that 
G(o,',r) is obtained from G^ by projection w.r.t. o,. 

For any weight function w'; £ —>■ Z, the reweighting of F = (y,E,w, (Vb,Fi)) w.r.t. w' is the 
arena F*^ = {V,E ,w' ,{Vo,Vi)). Also, for w.E^'L and any v ^'L,snq denote by w + V the weight 
function w' defined as =We + V. Indeed, we shall consider reweighted games of the form 

for some q €Q. Notice that the corresponding weight function w': F —^Q:ei—)-We—qis 
rational, while we required the weights of the arcs to be always integers. To overcome this issue, 
it is sufficient to re-define by scaling all weights by a factor equal to the denominator of 
q GQ; i.e., when q GQ, say q =N/D for gcd{N,D) = 1 we define = pD w-A' rescal¬ 
ing operation doesn’t change the winning regions of the corresponding games, let’s denote this 
equivalence as F”’^'? = and it has the significant advantage of allowing for a discussion 

(and an algorithmics) which is strictly based on integer weights. 


2.7. Mean Payoff Games 

_ A Mean Payoff Game (MPG) ( Brim et al. , 2011 ; Zwick and Paterso^ 1996t Ehrenfeucht and Mvcielski , 


19791) is a game played on some arena F for infinitely many rounds by two opponents. Player 0 


gains a payoff defined as the long-run average weight of the play, whereas Player 1 loses that 
value. Formally, the Player O’s payoff of a play vqVi ... v„ ... in F is defined as follows: 


MPo(vovi ...v„...) =liminf- Y w(v,-,v,+ i). 

fi 
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The value secured by a strategy Oo S 2o in a vertex v is defined as: 


val°“(v) = inf MPo(outcome''(v, ab7 0’i))) 

criGli 


Notice that payoffs and secured values can be defined symmetrically for the Player 1 (i.e., by 
interchanging the symbol 0 wit h 1 and inf with sup). 


Ehrenfeucht and Mycielski lEhrenfeucht and Mvcielskil (Il979h proved that each vertex v GV 
admits a unique value, denoted val'"(y), which each player can secure by means of a memoryless 
(or positional) strategy. Moreover, uniform positional optimal strategies do exist for both players, 
in the sense that for each player there exist at least one positional strategy which can be used to 
secure all the optimal values, independently with respect to the starting position v^. Thus, for 
every MPG T, there exists a strategy do G such that V'’'='"val°^“(v) > val'"(v), and there 
exists a strategy di G such that V'’'='^val‘^'(v) < val''(v). The (optimal) value of a vertex 
V GV in the MPG T is given by: 

val’"(v) = sup val°'’(v) = inf val‘^‘(v). 

0-0 gEo 


Thus, a strategy do G Eq is optimal if val^o^y) = val^(v) for all v GV. We denote optpE^ = 
{do e Ey(r) I val{{jj(v) = val’'(v)}. A strategy do G Eo is said to be winning for Player 0 


if V''^'^val°o(v) > 0, and di G Ei is winning for Player 1 if val°^' (v) < 0. Correspondingly, a 
vertex v G V is a winning starting position for Player 0 if val^(v) > 0, otherwise it is winning 
for Player 1. The set of all winning starting positions of Player i is denoted by Wi for / G | 0,1}. 
A refined formulation of the determinacy theorem is offered in Biorklund et al. ( 20041) . 


Theorem 1 ( Biorklund et al. ( 20041) ). Let T be an MPG and let be a partition (called er- 

godicj of its vertices into m>\ classes each one having the same optimal value V; G Q. Formally, 


V = U“ 1 Q andV‘^i'’^W'’^‘^' val^' (v) = Vi, where T,- 4 r|^_. 

Then, Player 0 has no vertices with outgoing arcs leading from Ci to Cj whenever Vi < w, 
and Player 1 has no vertices with outgoing arcs leading from Ci to Cj whenever Vi > Vji 
moreover, there exist do G E^ and G\ G E{^ such that: 

- If the game starts from any vertex in Ci, then do secures a gain at least Vi to Player 0 and 
di secures a loss at most v,- to Player 1; 

-Any play that starts from Ci always stays in Ci, if it is consistent with both strategies do, di, 
i.e., if Player 0 plays according to do, and Player 1 according to di. 


A finite variant of MPGs is well-known in the literature ( Ehrenfeucht and Mycielski, 19791: 
Zwick and Paterson . 19961: Brim et al. . 2011 ). where the game stops as soon as a cyclic se¬ 


quence ofverticesjsjraversed^lt turns out that this is equivalent to the infinite game formu¬ 
lation ( Ehrenfeucht and Mycielski, 1979h . in the sense that the values of an MPG are in a strong 
relationship with the average weights of its cycles, as in the next lemma. 


Proposition 1 (Brim, et al. Brim et all ( 2011 )). Let T be an MPG. For all V G Q, for all do G Ej^, 
and for all v G V, the value val^°(v) is greater than V iff all cycles C reachable from v in the 
projection graph G^^ have an average weight w(C)/|C| greater than v. 


The proof of Proposition [T] follows from the memoryless determinacy of MPGs. We remark 
that a proposition which is symmetric to Proposition [1] holds for Player 1 as well: for all V G Q, 
for all positional strategies di G E{^ of Player 1, and for all vertices v GV, the value val°^* (v) is 
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less than v iff if all cycles reachable from v in the projection graph have an ave r age w eight 
less than v. Also, it is well-known ( Brim et ^ 201 it Ehrenfeucht and Mvcielski . 1979h that 
each value val’"(v) is contained within the following set of rational numbers: 

Sr=\^N/D\D€[l,\V\],Ne[-D-W,D-W]y 


Notice, |5r| < |VpVk. 

The present work focuses on the algorithmics of the following classical problem: 

- Optimal Strategy Synthesis. Compute an optimal positional strategy for Player 0 in E. 

Also, in Section|5]we shall consider the problem of computing the whole optpE^: 

- Optimal Strategy Enumeration. Provide a listing of all the optimal positional strategies of 
Player 0 in the MPG E. 

2.2. Energy Games and Small Energy-Progress Measures 

An Energy Game (EG) is a game that is played on an arena E for infinitely many rounds by 
two opponents, where the goal of Player 0 is to construct an infinite play vqVi ...v„... such that 
for some initial credit c S N the following holds: c + ^(^qw(v,,v,+i) > 0, for all j > 0. Given 
an initial credit c G N, a play vqvi ... v„ ... is winning for Player 0 if it satisfies (1), otherwise 
it is winning for Player 1. A vertex v G V is a winning starting position for Player 0 if there 
exists an initial credit c G N and a strategy Ob G such that, for every strategy a\ G Ei, the play 
outcome^f v, Oh, <7i) is winni ng for Player 0. As in the case of MPGs, the EGs are memoryless 
determined Brim et all ( 2011 ), i.e., for every v G V, either v is winning for Player 0 or v is winning 
for Player 1, and (uniform) memoryless strategies are sufficient to win the game. In fact, as shown 
in the next lemma, the decision problems of MPGs and EGs are intimately related. 


Proposition 2 (IBrim et al.l (1201 Ih l. Let E be an arena. For all threshold V G Q, for all vertices 
V G E, Player 0 has a strategy in the MPG E that secures value at least V from v if and only if 
for some initial credit c G N, Player 0 has a winning strategy from v in the reweighted EG E“’^''. 

In this work we are especially interested in the Minimum Credit Problem (MCP) for EGs: for 
each winning starting position v, compute the minimum initial credit c* = c*(v) such that there 
exists a winning strategy Oq G for Player 0 startin g from v. A fast p seudo-polynomial time 


deterministic procedure for solving MCPs comes from Brim et al] ( 201lh . 

Theorem 2 dBrim et al.l (l201 1 )). There exists a deterministic algorithm for solving the MCP 


within <9(|y| lElfE) pseudo-polynomial time, on any input EG {y,E,w, (Vb,Ei))- 

The algorithm mentioned in Theorem|2]is the Value-Iteration algorithm dBrim et al. . 2011 1. 
Its rationale relies on the notion of Small Energy-Progress Measures (SEPMs). 


2.3. Energy-Lattices of Small Energy-Progress Measures 

Small-Energy Progress Measures are bounded, non-negative and integer-valued functions 
that impose local conditions to ensure global properties on the arena, in particular, witnessing 
that Player 0 has a way to enforce conservativity (i.e., non-negativity of cycles) in the resulting 


*The listing has to be exhaustive (i.e., each element is listed eventually) and without repetitions (i.e., no element is 
listed twice). 
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game’s graph. Recovering standard notation, see e.g. lBrim et al.l (1201 ih . let us denote = {« G 
N I n < (|y I — l)Vy} U {T} and let ^ be the total order on defined as: x ^ y ijf either y = T 
or x,y G N and x <y. In order to cast the minus operation to range over let us consider an 
operator Q : x Z ^ defined as follows: 


qq^a/ inax(0,a —Z?), if a ^ T and a — < (|y | — 1)1^; 
aQb=T, otherwise. 

Given an EG E on vertex set y = yo U yi, a function f :V ^ is a Small Energy-Progress 
Measure (SEPM) for E if and only if the following two conditions are met: 

1. if V £ Eo, then /(v) ^ /(v') © w(y, v') for some (v, v') G £; 

2. if V £ El, then /(v) ^ /(v') 0 w(v, v') for all (v, v') £ E. 

The values of a SEPM, i.e., the elements of the image /(E), are called the energy levels of /. 
It is worth to denote by Ey = {v £ E | f{v) ^T} the set of vertices having finite energy. Given a 
SEPM f :V and a vertex v £ Eq, an arc (v, v') £ £ is said to be compatible with / whenever 
/(r) ^/(v') ©w(v,v'); otherwise (v,v')is said lo incompatible with f. Moreover, a positional 
strategy Ob G is said to be compatible with / whenever: if <7o(y) = v' then (v, v') G £ is 

compatible with /; otherwise, Ob is incompatible with /. 

It is well-known that the family of all the SEPMs of a given E forms a complete (finite) 
lattice, which we denote by <Sy call it the Energy-Lattice of E. Therefore, we shall consider: 

^r=({/:E^'^r|/is SEPMofE},p, 


where for any two SEPMs f,g define f IZ g iff \/v € Vf{v) < g(v). Notice that, whenever / and 
g are SEPMs, then so is the minimum function defined as: \/'"^'^h{v) = min{/(v),g(v)}. This fact 
allows one to consider the least SEPM, namely, the unique SEPM f* :V such that, for any 
other SEPM g : E —the following holds: V'''=^/*(v) © g(v). Thus, <or is a complete lattice. 
So, (Sy enjoys of Knaster-Tarski Theorem, which states that the set of fixed-points of a monotone 
function on a complete lattice is again a complete lattice. 

Also concerning SEPMs, we shall rely on the following lemmata. The first one relates 
SEPMs to the winning region Wq of Player 0 in EGs. 


Proposition 3 ( Brim et ^ ( 2011 )1. Let E be an EG. Then the following hold. 


1. If f is any SEPM of the EG E and v £ Vf, then v is a winning starting position for Player 0 
in the EG E. Stated otherwise, Vf C 1^; 

2. If f* is the least SEPM of the EG E, and v is a winning starting position for Player 0 in 
the EG E, then v £ Vf*. Thus, Vf* = #o. 


The following bound holds on the energy-levels of any SEPM (by definition of %:). 


Proposition 4. Let E be an EG. Let f be any SEPM ofT. 

Then, for every v £ E either f(y) = T orO < f{v) < (|E| — 1)1E. 


3. Optimal Strategies from Reweightings 

It is now recalled a sufficient condition, for a positional strategy to be optimal, which is 
expressed in terms of reweighted EGs and their SEPMs. 
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Theorem 3 (IComin and Rizzil (l2016ah \ Let F = (y,E,w, (Vo, Vi)) be an MPG. For each u €V, 
consider the reweighted EG r„ = pgf ^ /jg ayiy SEPM o/r„ such that 

u £ (i.e., fu{u) 7 ^ Tj. Moreover, we assume: fu^ = fu 2 whenever val^{ui) = valF{u 2 ). 

When u £ Vo, let Vf^ £ Ny‘‘{u) be any vertex such that {u,Vf^) £ E is compatible with fu in 
EG r„, and consider the positional strategy (Tq £ defined as follows: <^0 (“) — Vf„- 

Then, (Jq is an optimal positional strategy for Player 0 in the MPG F 

Proof. See the proof of rTheorem 4 in Comin and Ri^ ( 2016a )1. □ 


Remark 1. Notice that Theorem |5] holds, particularly, when /„ is the least SEPM f* of the 
reweighted EG F„. This is because u £ Vf* always holds for the least SEPM f* of the EG F„; 
indeed, by Proposition^and by definition ofTu, then u is a winning starting position for Player 0 
in the EG F„ (for some initial credit); thus, by Proposition^ it follows that u £ Vf*. 


4. An Energy-Lattice Decomposition of optpE;^ 

Recall the example arena Fex shown in Fig. [1] It is easy to see that V'’'='^val'"“(v) = — 1. 
Indeed, Fex contains only two cycles, i.e.. Cl = [A,B,C,D] and Cr = [F,G], also notice that 
w{Cl)ICl = w{Cr)/Cr = — 1. The least-SEPM f* of the reweighted EG F’^^* can be computed 
by running a Value Iteration dBrim et al. , 201 ih . Taking into account the reweighting w'^w+l, 
as in Fig.ia f*{A) = f*{E) = f*{G) = 0, f*(B) = f*[D) = f*(F) = 4, and /*(C) = 8. 





Figure 2: The least-SEPM f* of Fgx+* (energy-levels are depicted in circled boldface). All and 
only those arcs of Player 0 that are compatible with f* are (B,C), (D,A), (E,A), (E,G), (G,F) 
(thick red arcs). 

So, Fex (Fig. IS implies the following. 

Proposition 5. The converse statement of Theorem |5] doesn ’t hold; there exist infinitely many 
MPGs F having at least one Ob £ optY'T.Q which is not compatible with the least-SEPM o/F. 

Proof. Consider the Fex of Fig. and the least-SEPM f* of the EG F^+'. The only vertex 
at which Player 0 really has a choice is E. Every arc going out of E is optimal in the MPG 
Fex; whatever arc {E,X) £ E (for any X £ {A,C,F,G}) Player 0 chooses at E, the resulting 
payoff equals val'^“(£') = — 1. Let f* be the least-SEPM of f* in Fgx+'. Observe, (E,C) and 
{E,F) are not compatible with f* in Fg(;+', only {E,A) and {E,G) are. Eor instance, the po¬ 
sitional strategy Oq G defined as Oo(£’) = F, (Jo(B) = C, Oo(D) = A, Oo(G) = F ensures a 
payoff V'''=^val’'“(v) = —1, but it is not compatible with the least-SEPM f* of Fg(+* (because 
f*{E) = 0 < 3 = f*{F) Qw(E,F)). It is easy to turn the Fex of Fig.|2]into a family on infinitely 
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many similar examples. 


□ 


We now aim at strengthening the relationship between optpE^ and the Energy-Lattice rot- 
For this, we assume w/og = v; this follows from Theorem [1] which allows 

one to partition T into several domains T; = Tj^ each one satisfying: = V,-. 

By Theorem[T]we can study optp independently w.r.t. optpX^ for J ^ i. 

We say that an MPG T is v-valued if and only if 3''^‘®V'’^'^val^(v) = V. 

Given an MPG T and Ob G E^(r), recall, G(r,do) — (YiE',w') is obtained from G'" by 
deleting all and only those arcs that are not part of Gq, i.e., 

£' = {(m,v) e £ I M e Vb and v = O'o(m)} U {(m,v) € £ | m e Vi}, 

where each e G £' is weighted as in T, i.e., w': E' ^ T,: e ^ Wg. 

When G = {V,E,w) is a weighted directed graph, a feasible-potential (FP) for G is any map 
7t :V ^ 't^G s-t. X ?t(v) © w(m, v). The least-FP n* = TIq is the (unique) FP 

s.t., for any other FP n, it holds V'''=^;r*(v) © 7t(v). Given G, the Bellman-Ford algorithm can 
be used to produce Kq in G(|y||£|) time. Let be the least-FP of G(r,Ob). Notice, for 

every Gq G E^, the least-FP is actually a SEPM for the EG F; still it can differ from the 

least-SEPM of F, due to Ob- We consider the following family of strategies. 

Definition 1 (AQ^(/,F)-Strategies). Let F = {V,E,w,{Vo,Vi)) and let f : V ^ be a SEPM 
for the EG F. Let Aq^(/,F) C Eq^(F) be the family of all and only those positional strategies of 
Player 0 in F s.t. coincides with f pointwisely, i.e., 

A"(/,F) 4 {(To G Ef(F) I y^*Gir,a,)iy) = 

We now aim at exploring further on the relationship between <fp and optpE^, via A^{f,r). 

Definition 2 (The Energy-Lattice of optpE^). Let F be a v-valued MPG. Let SG C Apw-v be a 
sublattice ofSEPMs of the reweighted EG F’^^''. 

We say that SG is an “Energy-Lattice of op ” iff\/-^^^A^{f,F^''^'') f 0 and the follow¬ 

ing disjoint-set decomposition holds: 

opt^I,^= □ A"(/,F--''). 

fear 

Lemma 1. £ef F be a v-valued MPG, and let cJq G opfpE^. Then, G(F“’^'',(7q) is conservative 
( i.e., it contains no negative cycle). 

Proof. Let C = (vi ...,Vk,v\) by any cycle in G(F*''’^'',cJq). Since we have Gq G optpEj^ and 
V'’^'^val’'(v) = V, thus w{C)/k = Vj+i) > V (for = vi) by PropositionlT] so that, 

assuming w' = w — V, then: w'{C)/k — ^Lf=i ('^(FWi+i) ~ v) = w{C)/k — v > V — V = 0. □ 

Some aspects of the following Proposition rely heavily on Theorem |3] the compatibility 
relation comes again into play. Moreover, we observe that Proposition |6] is equivalent to the 
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following fact, which provides a sufficient condition for a positional strategy to be optimal. Con¬ 
sider a v-valued MPG F, for some V G Q, and let cJq G optpE^. Let do G E^(r) be any (not 
necessarily optimal) positional strategy for Player 0 in the MPG F. Suppose the following holds: 

Then, by Proposition 0 do is an optimal positional strategy for Player 0 in the MPG F. 

We are thus relying on the same compatibility relation between and SEPMs in reweighted 
EGs which was at the base of Theorem|3l aiming at extending TheoremOso to describe the whole 
optpE^ (and not just the join/top component of it). 

Proposition 6. Let the MPG F be v-valued, for some V G Q. 

There is at least one Energy-Lattice of op fpE^.' 

Jfp* = I C7o G optpE^}. 

Proof The only non-trivial point to check being: A^(/,F’*’^'') C optpE^. 

For this, we shall rely on Theorem[3 Let / G G ^oif^ F“'^'') be fixed (arbitrarily). 

Since / G then / = 7rgjp„_v g-*) for some cTq G optpE^. Therefore, the following holds: 

Clearly, do is compatible with / in the EG because / = 7rQ^p„,_v By Lemma[T] since 
(Tq is optimal, then G(F*’'^'', cJq ) is conservative. Therefore: 


Notice, do satisfies exactly the hypotheses required by Theorem |3] Therefore, do G optpE^. 
This proves (*).This also shows optpEj^ = U/eXp ^rid concludes the proof. □ 


Proposition 7. Let the MPG F be v-valued, for some V G Q. Let \ Energy- 

Lattices for optYL-Q . Then, 3 Gy ^ = ^y 2 - 

Proof By symmetry, it is sufficient to prove that ^y \ — ^T 2 - ^ \ fixed (arbitrar¬ 

ily). Then, /i = ;r^(pH.-v_^g) for some do G optpE^. Since do G optpE^ and since ifi^Y 2 ™ 

Energy-Lattices, there exists /2 G S^y-i s.t. do G Ay(/ 2 ,F“'^''), which implies n^^Y^'-v ^ 0 ) = /z- 
Thus, /i = ?re(p»-v a-o) = /2- This implies /i G Jfp □ 

The next theorem summarizes the main point of this section. 

Theorem 4. Let T be a V -valuedMPG, for some v gQ. Then, SG-f — {^g(P'-'' oq) I *^0 ^ °pfr^o^} 
is the unique Energy-Lattice of optpE^. 

Proof By Proposition|6]and Proposition [T] □ 
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Example 1. Consider the MPG Tex, as defined in Fig. |7] Then, where f* 

is the least-SEPM of the reweighted EG and where the following holds: fi (A) = / 2 (A) = 
/* (A) = 0; /i (B) = /2 (B) = r (B) = 4; f,{C)= /z (C) = /* (C) = 8; /i (D) = /z (D) = f* (D) = 
4; /i(E) = / 2 (B) = r(B) = 4; /i(G) = / 2 (G) = /*(G) = 0; finally, f*{E) = 0, /i(B) = 3, 
fliE) = 7. A« illustration of f\ is offered in Fi 2 . \3a\ ( enerev-levels are depicted in circled bold¬ 
face). whereas f 2 is depicted in Fig. \3^ Notice that f* (v) < f\ (v) < / 2 (v) for every v CV, and 
this ordering relation is illustrated in Fig. |i] 



(a) The extremal-SEPM /i of r"+> (b) The extremal-SEPM B of . 


Definition 3. Each element f € called extremal-BBBM. 

The next lemma is the converse of Lemma[T] 

Lemma 2. Let the MPG T be V-valued, for some V S Q. Consider any <7o S Ej^(r), and assume 
that G{r'''^^, Go) is conservative. Then, Oq G optYT^. 

Proof. Let C = (vi,..., v^vi) any cycle in G(r,ao). Then, the following holds (if = vi): 
^ = |Lf=iVP(PoP/+i) = v + jLf^i (w(y;,v,+i)- > V, where (w(v/,v,+i)- v) >0 

holds because G(r'^’^'',00) is conservative. By Proposition[T] since w(C) l£>V for every cycle 
C in G^g, then do S optpL^. □ 


The following proposition asserts some properties of the extremal-SEPMs. 

Proposition 8. Let the MPG T be V-valued, for some V G Q. Let IXf-f be the Energy-Lattice of 
op Moreover, let f :V ^ a SEPMfor the reweighted EG T’*^''. Then, the following 
three properties are equivalent: 

1 . fG3F^; 

2. There exists Go G opt^T^ s.t. “ /('') for every v GV. 

3. Vf = Wo{T^-'') = V andA^(f,r^'-'') f 0; 

Proof of (I 2). Indeed, JTp* = {Ttgjrw-v | O-q S optpL^}. □ 

Proof of (I ^3). Assume / e Since (1 2), there exist do G optpL^ s.t. = 

/. Thus, do e A^{f,T"’-^), so that f 0. We claim Vf = Wo{T*'-'') = V. Since 

V(v G T)val^(v) = V, then = V by Proposition|2l Next, G(P''’^'', do) is conservative 
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ff\B)=C 


af>{E) = C 

<yf\G) = F 


AS'(/i,r“’-'') = {a«} 



af\E) = F 

<T®(G) = f 



A«(ppr-'') = {<T<‘>,<TP>} 


II 

o 

4^\d)=a 

Gf\D)=A 

4'\e)=a 

Gf\E)=G 

<7™(G) = f 

Gf\G)=F 



Figure 3; The decomposition of optpE^ (right), for the MPG Fex, which corresponds to the 
Energy-Lattice = {/*,/i,/ 2 } (center) (as in Example[T]). Here, /* < /i < fi- This brings 
a lattice &Ycx ^ basic subgames of Fex (left). 


by Lemma [T] Since G(r*‘' '' 
Vf = iro(r“'-'') = y. 


, Ob) is conservative and / = 7rg^p,,_v then Vf = V . Therefore, 

□ 


Proof of (I <= 3). Since Ay(/,r“' '') f 0, pick some Oo G A^{f,r'^ so, / = Tt^^Y"-'',aoy 
Since Vf = V and / = ;rg^p„,_v then G(r'^^'', Oo) is conservative. Since G(r*^’^'', Oo) is con¬ 
servative, then Oq G optpEo^ by Lemma ^ Since / = Tt^, and Oq G optpE^, then / G ^f^Y 
because 2 1. □ 


5. A Recursive Enumeration of S^y and optp (E“) 


An enumeration algorithm for a set S provides an exhaustive listing of all the elements of S 
(without repetitions). As mentioned in Section ID by Theorem[T] no loss of g enerality occurs if 
we assume F to be v-valued for some V G Q. One run of the algorithm given in IComin and Rizzi 


(12016H) allows one to partition an MPG F, into several domains F, each one being v,-valued 
for V, G 5r; in GdEplLliy) time and linear space. Still, by Proposition |5] Theorem[3is not 
sufficient for enumerating the whole optp)^^); it is enough only for where /y is 
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the least-SEPM of which is just the join/top component of optp ). However, thanks to 
TheoremlH we now have a refined description of optpSj^ in terms 5^y ■ 

We offer a recursive enumeration of all the extremal-SEPMs, i.e., and for computing 
the corresponding partitioning of optp ). In order to avoid duplicate elements in the enumer¬ 
ation, the algorithm needs to store a lattice of subgames of F, which is related to ■ We 
assume to have a data-structure Fp supporting the following operations, given a subarena F' of 
F: insert(F', Tf) stores F' into Fp; contains(F', Fp) returns T if and only if F' is in Fp, and F 
otherwise. A simple implementation of Fp goes by indexing A™*(v) for each v € V (e.g., with a 
trie data-structure). This can run in OdFl log |y |) time, consuming OdFl) space per stored item. 
Similarly, one can index SEPMs in (9(|y |logdy|iy)) time and (9(|y |) space per stored item. 

The listing procedure is named enumO, it takes a v-valued MPG T and goes as follows. 

1. Compute the least-SEPM /* of F, and print F to output. Theorem[3]can be employed at 
this stage for enumerating A^(/*,F“'^''): indeed, these are all and only those positional 
strategies lying in the Cartesian product of all the arcs (m, v) € E that are compatible with 
f* in F”'-'' (because f* is the least-SEPM of E). 

2. Let St 4— 0 be an empty stack of vertices. 

3. Eor each u G Vq, do the following: 

• Compute Eh 4— {{u,v) GE \ f*(u) ^ /*('') © {w{u,v) — v)}; 

• YfEa^ 0, then: 

- Let £' 4— Fa U{(m, v) GE \ u^u} and F' 4— {V.E' (yo,yi)). 

- If contains(F', Fp) = F, do the following: 

4= Compute the least-SEPM f* of F'”'^''; 

* Ify^,* =y: 

- Push u on top of St and insert(F',Fp). 

-If contains(/'*, Fp) = F, then insert)/'*, Fp) and print/'*. 

4. While St/0: 

• pop u from St; Let 4— {(m, v) G £ | f*{u) < /*(v) © (w(m,v) — v)}, and E' 4— 
£(} U{(m, v) G £ I M / «}, and F' 4— (y,£',w, (yo,yi)); 

• Make a recursive call to enum () on input F'. 

Down the recursion tree, when computing least-SEPMs, the children Value-Iterations can amor¬ 
tize by starting from the energy-levels of the parent. The lattice of subgames .^p comprises 
all and only those subgames F' C F that are eventually inserted into Fp at Step (3) of enum(); 
these are called the basic subgames of F. The correctness of enumO follows by Theorem|4]and 
Theorem^ In summary, we obtain the following result. 

Theorem 5. There exists a recursive algorithm for enumerating (w/o repetitions) all elements 
of^Y with time-delaj^O{\V\^\E\W), on any input MPG F; moreover, the algorithm works with 
6>dy||£|)-|-0(|£||,^p|)) space. So, it enumerates iw/o repetitions) in (9(|yp|£|iy|.i^p|) 
total time, and(9dy||£|)-|-0(|£||,^p|) space. 


listing algorithm has 0{f{n)) time-delay when the time spent between any two consecutives is 0{f{n)). 
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To conclude we observe that and ^re not isomorphic as lattices, not even as sets 
(the cardinality of can be greater that that of Indeed, there is a surjective antitone 

mapping (pr from onto (i.e., (pr sends T' G ^p to its least-SEPM /p, G still, we 
can construct instances of MPGs such that |^p| > i.e., (pr is not into and not 

isomorphic. That would be a case of degeneracy, and an example MPG Td is given in Fig.|4l 



Figure 4; An MPG Fd for which | > | JTp* |. 




Figure 5; Two basic subgames F^ ^ F^ of Fd, having the same least-SFPM /j* = /|. 


In the MPG Fd, Player 0 has to decide how to move only at M3,V3 and f; the remain¬ 
ing moves are forced. The least-SFPM f* of Fd is: f*{ui) = 1, f*{vj) = 1, /*(f) = 0, and 
f* (■^) = Oi leading to the following memory-less strategy: cTq (M 3 ) = f, cJq (V 3 ) = f, 
( 7 q (f ) = V4. Then, consider the lattice of subgames ^p^; particularly, consider the following two 
basic subgames F^, F^: let be the arena obtained by removing the arc (f, V4) from F^; let F^ be 
the arena obtained by removing the arc from F)^; let F^ be the arena obtained by removing 
the arc (v 3 ,f) from See Fig.|5]for an illustration. Next, let be the least-SFPMs of F^ 

and Y% respectively; then, /f (M 3 ) = (M 3 ) = 2, /f (V 3 ) = (vs) = 2, /f (f) = /j (f) = 10, and 

''^jtGVrj\{« 3 ,v 3 .r}/r(■^) = fi (^) = Thus, F^ 7 ^ F^, but /j* = /|; this proves that Fd is degenerate 
and that ,^p, not isomorphic. 
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6. Conclusion 


We observed a unique complete decomposition of optpE^ in terms of extremal-SEPMs in 
reweighted EGs, also offering a pseudo-polynomial total-time recursive algorithm for enumer¬ 
ating (w/o repetitions) all the elements of i.e., all extremal-SEPMs, and for computing the 
components of the corresponding partitioning of optpEj^. 

It would be interesting to study further properties enjoyed by i^p and and we ask for 
more efficient algorithms for enumerating e.g., pseudo-polynomial time-delay and polyno¬ 
mial space enumerations. 
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